Testing Task Force Quarterly Update

This article originally appeared in The Bar Examiner print edition, Summer 2019 (Vol. 88, No. 2), pp 37–38.

For the latest Testing Task Force news—and to sign up to receive updates with the latest Task Force news, research, and blog posts—visit www.testingtaskforce.orgTesting Task Force Launches Nationwide 2019 Practice Analysis

The Testing Task Force has been busier than ever: it published its First Year Report in May, completed Phase 1 research in June, and is currently in the midst of Phase 2, which consists of collecting empirical data regarding the job tasks of newly licensed lawyers through a nationwide 2019 practice analysis survey that launched on August 1.

First Year Report Now Available

The First Year Report, available at www.TestingTaskForce.org, provides an overview of the Task Force’s activities in 2018—from initial planning to the selection of research consultants, the establishment of a timeline for its three-year study, and the beginning of Phase 1 (stakeholder listening sessions), which has now concluded. The report highlights all the work that has been accomplished and what is in the pipeline.

Stakeholder Listening Sessions Completed

The stakeholder listening sessions were facilitated by ACS Ventures LLC (ACS) from November 2018 through June 2019, and an executive summary of Phase 1 will be available soon. Over 400 people from across the country participated in 30 in-person and virtual sessions, during which ACS gathered feedback on NCBE’s current tests that are used by a majority of U.S. jurisdictions for admission to their bars: the Multistate Professional Responsibility Examination (MPRE), the Multistate Bar Examination (MBE), the Multistate Essay Examination (MEE), and the Multistate Performance Test (MPT).

Photo of a facilitator standing and talking to a group of seated people

Listening sessions held at the 2019 NCBE Annual Bar Admissions Conference

Photo of a participant at a focus group session speaking into a microphone

Listening sessions held at the 2019 NCBE Annual Bar Admissions Conference

The facilitators solicited opinions concerning what aspects of the current tests stakeholders like and dislike, what characteristics they would want to see in the next generation of the bar exam, and what cautions should be considered regarding potential changes. Discussions centered around the content, format, timing, and delivery method of the bar exam, and all feedback—whether it was positive or negative—was welcome. Interestingly, many themes resonated across sessions and various stakeholder groups (e.g., an overwhelming endorsement of the MPT), but there were likewise many diverging opinions ranging from people largely supporting the current bar exam to those suggesting significant changes. The Task Force thanks everyone who took the time to share their thoughtful insights by participating in a session. The feedback garnered from Phase 1 provides the Task Force with much to consider as it starts to explore various test design options for the future bar exam.

Practice Analysis Survey Launched August 1

Phase 2 of the Task Force’s study, which has been under way for several months, culminated in the launch of a nationwide practice analysis survey on August 1, 2019. The purpose of this practice analysis is to collect current information on the job tasks performed by newly licensed lawyers and the knowledge, skills, and abilities required to accomplish those tasks, as well as the technology used in law practice.

To ensure that the survey results in a representative data set that reflects the responses of attorneys from a breadth of practice areas and settings, the practice analysis is being broadly disseminated. The Task Force enlisted bar associations and state Supreme Courts to assist in distributing the practice analysis survey to give all attorneys the opportunity to answer it. The Courts and bars have overwhelmingly agreed to help with this endeavor, and the Task Force is extremely appreciative of their assistance.

The results of the practice analysis will be used by the Task Force to make certain that the bar exam continues to test appropriate content in a rapidly changing legal profession. But beyond that, the data will also be informative to all stakeholders (law students and schools, employers, bar admissions authorities, the public, etc.) interested in the work that newly licensed lawyers perform and the associated competencies needed to enter practice. All attorneys are encouraged to share their insights by taking the survey, which will be open through the end of September. Results will be published early next year.

The Task Force Thanks Jurisdictions Helping to Distribute the 2019 Practice Analysis Survey (as of July 31, 2019)

Map of the United States show jurisdictions distributing the practice analysis survey. All jurisdictions are distributing except Illinois, Louisiana, and Puerto Rico

Phase 3 of the Study to Begin at End of Year

In the interim, Phase 3—the last and most challenging component of the Task Force’s study—will kick off at the end of this year. The research from Phases 1 and 2 will provide the foundation to inform all decisions regarding the bar exam of the future, and stakeholders will have the opportunity to weigh in on the various test design options the Task Force will contemplate.

Visit the Task Force Website!

To access the Testing Task Force First Year Report and the Executive Summary of Phase 1, and to take the 2019 Practice Analysis Survey, visit TestingTaskForce.org.

NCBE’s Testing Task Force

The National Conference of Bar Examiners’ Testing Task Force, appointed by NCBE in January 2018, is charged with undertaking a three-year study to ensure that the bar examination continues to test the knowledge, skills, and abilities (KSAs) required for competent entry-level legal practice in the 21st century. The study is scheduled to be completed by the end of 2020. The Testing Task Force’s study will be comprehensive, future-focused, collaborative, empirical, and transparent.